Skip to content

ci: test unsupported AOT runtimes in the build legs - #5811

Merged
ric-oliv merged 1 commit into
mainfrom
ci/test-unsupported-aot-in-build-legs
Oct 27, 2026
Merged

ric-oliv merged 1 commit into
mainfrom
ci/test-unsupported-aot-in-build-legs

Conversation

@ric-oliv

@ric-oliv ric-oliv commented Oct 25, 2026 •

Copy link
Copy Markdown
Member

Part 5 of 6 for #5806, based on #5810. The two Unsupported AOT jobs check that Sentry turns SentryNative off on runtimes without a native SDK, linux-arm and win-x86. They waited for the whole build matrix to test the macOS packages, and then added their own minutes after it. Any leg's packages work, though, and every leg already packs its own. So linux-arm now runs in the linux-arm64 leg and win-x86 in the win-x64 leg, and the two jobs are gone.

The linux-arm job was also flaky. 4 of 57 attempts failed, and the one log left showed "Exec format error", because some arm64 hosts can't run 32-bit ARM binaries.

  • The leg registers QEMU for 32-bit ARM with docker/setup-qemu-action, pinned as in alpine.yml.
  • Its binfmt_misc handler comes before the kernel's own loader, so QEMU runs the test app on every host, even where the host could run it natively. In ci: test unsupported AOT runtimes in the build legs #5791 a small uname program confirmed that, reporting armv7l on such a host.
  • The leg now checks only that the handler is enabled.

This adds one dependency: setup-qemu-action pulls tonistiigi/binfmt from Docker Hub on every linux-arm64 run, as alpine.yml already does. If Docker Hub is down, that leg fails at that step. The action runs that image privileged, so it's pinned by digest, here and in alpine.yml, the way the action itself is pinned.

In #5791 the tail after the matrix was gone. On linux-arm64 the new steps took about a minute, and the AOT and container cases passed under QEMU.

On this PR's own CI

3 runs. On linux-arm64 the four new steps took 1.0 minute each time, 0.6 of it the linux-arm AOT case, and the handler check passed every time. On win-x64 the win-x86 case took 0.5 to 0.8. No job waits for every leg anymore. On main, the two AOT jobs ran 1.7 minutes past the last leg. The whole stack's numbers are in #5806.

#skip-changelog

@codecov

codecov Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 75.93%. Comparing base (d4dc935) to head (cd1480d).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5811      +/-   ##
==========================================
- Coverage   76.13%   75.93%   -0.20%     
==========================================
  Files         528      528              
  Lines       19579    19579              
  Branches     3829     3829              
==========================================
- Hits        14906    14868      -38     
- Misses       3791     3824      +33     
- Partials      882      887       +5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ric-oliv
ric-oliv force-pushed the ci/test-unsupported-aot-in-build-legs branch from 1f30f05 to a807d74 Compare October 25, 2026 16:22
@ric-oliv
ric-oliv force-pushed the ci/run-device-integration-tests-in-own-jobs branch from 2a0f2f4 to 1634a0f Compare October 25, 2026 16:22
@ric-oliv
ric-oliv marked this pull request as ready for review October 26, 2026 19:05
@github-actions github-actions Bot added the risk: medium PR risk score: medium label Oct 26, 2026
@ric-oliv
ric-oliv force-pushed the ci/test-unsupported-aot-in-build-legs branch from a807d74 to 44331a8 Compare October 26, 2026 19:15
@ric-oliv
ric-oliv force-pushed the ci/run-device-integration-tests-in-own-jobs branch from 1634a0f to 3db1268 Compare October 26, 2026 19:15
path: integration-test

# Sentry turns SentryNative off on runtimes that have no native SDK. aot.Tests.ps1 checks that against this
# leg's own packages: linux-arm on the linux-arm64 runner, and win-x86 on the win-x64 runner.

@jamescrosswell jamescrosswell Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aot.Tests.ps1 checks that against this leg's own packages

Similar issue to https://github.com/getsentry/sentry-dotnet/pull/5809/changes#r4383085676 but probably not as much of a concern here as I think AOT builds/tests for non-mobile code are less sensitive to changes to the runners. So OK to do it in this case - just worth noting we'd be breaking an otherwise long held and inflexible rule that we only/always test the same thing that we ship.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. With the #5809 change, release runs test the shipped MAUI packages again. The unsupported AOT cases stay on the leg's own packages: whether SentryNative is on comes from a fixed RID list in the package's targets, not from what's bundled.

@ric-oliv
ric-oliv force-pushed the ci/run-device-integration-tests-in-own-jobs branch from 3db1268 to 4bf3776 Compare October 27, 2026 09:13
@ric-oliv
ric-oliv force-pushed the ci/test-unsupported-aot-in-build-legs branch from 44331a8 to 0552c5b Compare October 27, 2026 09:30

@bitsandfoxes bitsandfoxes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get the intention of the test and the benefit of guaranteeing that the fallback/backoff work. It just seems a bit odd to install packages and dependencies to run something somewhere it is not really supported, to then check that the thing that is not supported.. falls back to disabling something?

If you're happy, LGTM? 🤷‍♂️

@ric-oliv

ric-oliv commented Oct 27, 2026 •

Copy link
Copy Markdown
Member Author

Fair question! It's a regression test added in #4347 for a previous crash, so I'm currently only moving it away from the critical path.
Tagging @jamescrosswell here for suggestions but I'd anyway remove it in a separate PR if we come to that conclusion :)

@ric-oliv
ric-oliv force-pushed the ci/run-device-integration-tests-in-own-jobs branch from 4bf3776 to 050b256 Compare October 27, 2026 12:22
Base automatically changed from ci/run-device-integration-tests-in-own-jobs to main October 27, 2026 12:22
The Unsupported AOT jobs waited for the whole build matrix to use the
macOS packages, but any leg's packages work. Every leg already packs
and runs aot.Tests.ps1 against its own. linux-arm now runs in the
linux-arm64 leg and win-x86 in the win-x64 leg, which removes two jobs
and the tail after the matrix.

Some arm64 hosts can't run 32-bit ARM binaries and failed with "Exec
format error", so QEMU runs them on every host. Its binfmt_misc handler
comes before the kernel's own loader, so it runs them even on hosts that
could. On the first runs a small uname program confirmed that: it
reported armv7l, which only QEMU does, on a host that runs 32-bit ARM
natively. The leg now only checks that the handler is enabled. The
action runs the tonistiigi/binfmt image privileged, so that image is
pinned by digest here and in alpine.yml, like the action itself.

On its first run, the AOT and container cases passed under QEMU in 38
seconds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ric-oliv
ric-oliv force-pushed the ci/test-unsupported-aot-in-build-legs branch from 0552c5b to cd1480d Compare October 27, 2026 12:23
@ric-oliv
ric-oliv merged commit 7c4dc43 into main Oct 27, 2026
19 checks passed
@ric-oliv
ric-oliv deleted the ci/test-unsupported-aot-in-build-legs branch October 27, 2026 12:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk: medium PR risk score: medium

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants